Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable runtime config log level #19611

Merged

Conversation

Junchao-Mellanox
Copy link
Collaborator

@Junchao-Mellanox Junchao-Mellanox commented Jul 18, 2024

HLD link: sonic-net/SONiC#1522

Why I did it

SONiC provides two Python logger implementations: sonic_py_common.logger.Logger and sonic_py_common.syslogger.SysLogger. Both of them do not provide the ability to change log level at real time. Sometimes, in order to get more debug information, developer has to manually change the log level in code on a running switch and restart the Python daemon. This is not convenient.

SONiC also provides a C/C++ logger implementation in sonic-platform-common.common.logger.cpp. This C/C++ logger implementation is also a wrapper of Linux standard syslog which is widely used by swss/syncd. It provides the ability to set log level on fly by starting a thread to listen to CONFIG DB LOGGER table change. SONiC infrastructure also provides the Python wrapper for sonic-platform-common.common.logger.cpp which is swsscommon.Logger. However, this logger implementation also has some drawbacks:

  1. swsscommon.Logger assumes redis DB is ready to connect. This is a valid assumption for swss/syncd. But it is not good for a Python logger implementation because some Python script may be called before redis server starting.
  2. swsscommon.Logger wraps Linux syslog which only support single log identifier for a daemon.

So, swsscommon.Logger is not an option too.

This PR is a Python logger enhancement which allows user setting log level at run time.

Work item tracking
  • Microsoft ADO (number only):

How I did it

swsscommon.Logger depends on a thread to listen to CONFIG DB LOGGER table change. It refreshes log level for each logger instances once the thread detects a DB entry change. A thread is considered heavy in a python script, especially that there are many short and simple python scripts which also use logger. To keep python logger light weight, it uses a different design than swsscommon.Logger:

  • A class level logger registry shall be added to SysLoggerclass
  • Each logger instance shall register itself to logger register if enables runtime configuration
  • Logger configuration shall be refreshed by CLI which send a SIGHUP signal to the daemon

How to verify it

Manual test
New unit test cases

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211
  • 202305

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Copy link
Contributor

@prgeor prgeor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xincunli-sonic can you review?

@Junchao-Mellanox
Copy link
Collaborator Author

Hi @xincunli-sonic , we had a offline discussion about the multi asic issue. I have already updated the CLI to handle multi ASIC configuration, that part is not in this PR. Could you please review and sign off this one?

@Junchao-Mellanox
Copy link
Collaborator Author

/azpw run ms_checker

@mssonicbld
Copy link
Collaborator

/AzurePipelines run ms_checker

Copy link

No pipelines are associated with this pull request.

@liushilongbuaa
Copy link
Contributor

/azpw ms_conflict -f

Copy link
Contributor

@xincunli-sonic xincunli-sonic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Junchao-Mellanox Junchao-Mellanox force-pushed the master-runtime-log-config branch from 3f0f5b9 to d81abd5 Compare August 14, 2024 01:37
@Junchao-Mellanox
Copy link
Collaborator Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@prgeor
Copy link
Contributor

prgeor commented Aug 26, 2024

@Junchao-Mellanox can you point to the HLD link in the PR description?

@Junchao-Mellanox
Copy link
Collaborator Author

@Junchao-Mellanox can you point to the HLD link in the PR description?

Added

@Junchao-Mellanox
Copy link
Collaborator Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@Junchao-Mellanox
Copy link
Collaborator Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@Junchao-Mellanox
Copy link
Collaborator Author

/azpw run Azure.sonic-buildimage

@mssonicbld
Copy link
Collaborator

/AzurePipelines run Azure.sonic-buildimage

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Contributor

@prgeor prgeor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liat-grozovik could you please merge

@Junchao-Mellanox
Copy link
Collaborator Author

Hi @liat-grozovik , could you please help merge?

@Junchao-Mellanox
Copy link
Collaborator Author

Hi @qiluo-msft , could you please help merge this PR? All reviewers have approved the PR.

@liat-grozovik liat-grozovik merged commit 47aa38c into sonic-net:master Oct 27, 2024
23 checks passed
@Junchao-Mellanox Junchao-Mellanox deleted the master-runtime-log-config branch October 29, 2024 10:04
rkavitha-hcl pushed a commit to rkavitha-hcl/sonic-buildimage that referenced this pull request Nov 15, 2024
HLD link: sonic-net/SONiC#1522

- Why I did it
SONiC provides two Python logger implementations: sonic_py_common.logger.Logger and sonic_py_common.syslogger.SysLogger. Both of them do not provide the ability to change log level at real time. Sometimes, in order to get more debug information, developer has to manually change the log level in code on a running switch and restart the Python daemon. This is not convenient.

SONiC also provides a C/C++ logger implementation in sonic-platform-common.common.logger.cpp. This C/C++ logger implementation is also a wrapper of Linux standard syslog which is widely used by swss/syncd. It provides the ability to set log level on fly by starting a thread to listen to CONFIG DB LOGGER table change. SONiC infrastructure also provides the Python wrapper for sonic-platform-common.common.logger.cpp which is swsscommon.Logger. However, this logger implementation also has some drawbacks:

swsscommon.Logger assumes redis DB is ready to connect. This is a valid assumption for swss/syncd. But it is not good for a Python logger implementation because some Python script may be called before redis server starting.
swsscommon.Logger wraps Linux syslog which only support single log identifier for a daemon.
So, swsscommon.Logger is not an option too.

This PR is a Python logger enhancement which allows user setting log level at run time.

- How I did it
swsscommon.Logger depends on a thread to listen to CONFIG DB LOGGER table change. It refreshes log level for each logger instances once the thread detects a DB entry change. A thread is considered heavy in a python script, especially that there are many short and simple python scripts which also use logger. To keep python logger light weight, it uses a different design than swsscommon.Logger:

A class level logger registry shall be added to SysLoggerclass
Each logger instance shall register itself to logger register if enables runtime configuration
Logger configuration shall be refreshed by CLI which send a SIGHUP signal to the daemon

- How to verify it
Manual test
New unit test cases
aidan-gallagher pushed a commit to aidan-gallagher/sonic-buildimage that referenced this pull request Nov 16, 2024
HLD link: sonic-net/SONiC#1522

- Why I did it
SONiC provides two Python logger implementations: sonic_py_common.logger.Logger and sonic_py_common.syslogger.SysLogger. Both of them do not provide the ability to change log level at real time. Sometimes, in order to get more debug information, developer has to manually change the log level in code on a running switch and restart the Python daemon. This is not convenient.

SONiC also provides a C/C++ logger implementation in sonic-platform-common.common.logger.cpp. This C/C++ logger implementation is also a wrapper of Linux standard syslog which is widely used by swss/syncd. It provides the ability to set log level on fly by starting a thread to listen to CONFIG DB LOGGER table change. SONiC infrastructure also provides the Python wrapper for sonic-platform-common.common.logger.cpp which is swsscommon.Logger. However, this logger implementation also has some drawbacks:

swsscommon.Logger assumes redis DB is ready to connect. This is a valid assumption for swss/syncd. But it is not good for a Python logger implementation because some Python script may be called before redis server starting.
swsscommon.Logger wraps Linux syslog which only support single log identifier for a daemon.
So, swsscommon.Logger is not an option too.

This PR is a Python logger enhancement which allows user setting log level at run time.

- How I did it
swsscommon.Logger depends on a thread to listen to CONFIG DB LOGGER table change. It refreshes log level for each logger instances once the thread detects a DB entry change. A thread is considered heavy in a python script, especially that there are many short and simple python scripts which also use logger. To keep python logger light weight, it uses a different design than swsscommon.Logger:

A class level logger registry shall be added to SysLoggerclass
Each logger instance shall register itself to logger register if enables runtime configuration
Logger configuration shall be refreshed by CLI which send a SIGHUP signal to the daemon

- How to verify it
Manual test
New unit test cases
bradh352 added a commit to bradh352/sonic-buildimage that referenced this pull request Nov 29, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
bradh352 added a commit to bradh352/sonic-buildimage that referenced this pull request Nov 29, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
bradh352 added a commit to bradh352/sonic-buildimage that referenced this pull request Dec 1, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
github-actions bot pushed a commit to bradh352/sonic-buildimage that referenced this pull request Dec 1, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
bradh352 added a commit to bradh352/sonic-buildimage that referenced this pull request Dec 1, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
github-actions bot pushed a commit to bradh352/sonic-buildimage that referenced this pull request Dec 3, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
github-actions bot pushed a commit to bradh352/sonic-buildimage that referenced this pull request Dec 3, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
github-actions bot pushed a commit to bradh352/sonic-buildimage that referenced this pull request Dec 4, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
github-actions bot pushed a commit to bradh352/sonic-buildimage that referenced this pull request Dec 4, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
github-actions bot pushed a commit to bradh352/sonic-buildimage that referenced this pull request Dec 5, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
github-actions bot pushed a commit to bradh352/sonic-buildimage that referenced this pull request Dec 6, 2024
As of sonic-net#19611
and sonic-net/sonic-utilities#3428
a new database field of `require_manual_refresh` was added.

This leads to YANG failures in the factory-default configuration
for the `xcvrd` log entry which by default is written as true.

Signed-off-by: Brad House (@bradh352)
StormLiangMS pushed a commit that referenced this pull request Dec 6, 2024
Why I did it
YANG failures during config replace with default configuration.

As of #19611 and sonic-net/sonic-utilities#3428 a new database field of require_manual_refresh was added.

This leads to YANG failures in the factory-default configuration for the xcvrd log entry which by default is written as true.

Work item tracking
How I did it
Updated YANG file for new field.

How to verify it
Verify config replace works with factory default configuration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants